Bayesian Shrinkage Variable Selection April 25 , 2008

We introduce a new Bayesian approach to the variable selection problem which we term Bayesian Shrinkage Variable Selection (BSVS ). This approach is inspired by the Relevance Vector Machine (RVM ), which uses a Bayesian hierarchical linear setup to do variable selection and model estimation. RVM is typically applied in the context of kernel regression although it is also suitable in the standard regression context. Extending the RVM algorithm, we include a proper prior distribution for the precisions of the regression coefficients, v j ∼ f(v −1 j |η), where η is a scaler hyperparameter. Based upon this model, we derive the full set of conditional distributions for parameters as would typically be done when applying Gibbs sampling. However, instead of simulating samples from the joint posterior distribution in order to estimate the posterior means of the parameters, we use the full conditionals in order